Iteration (machine learning)
A single update of a model’s parameters—the model’s weights and biases—during training. The batch size determines how many examples the model processes in a single iteration. For instance, if the batch size is 20, then the model processes 20 examples before adjusting the parameters. When training a neural network, a single iteration involves the following two passes:1
- A forward pass to evaluate loss on a single batch.
- A backward pass (backpropagation) to adjust the model’s parameters based on the loss and the learning rate.